Low-Dimensional Manifold Distributional Semantic Models

Motivated by evidence in psycholinguistics and cognition, we propose a hierarchical distributed semantic model (DSM) that consists of low-dimensional manifolds built on semantic neighborhoods. Each semantic neighborhood is sparsely encoded and mapped into a low-dimensional space. Global operations are decomposed into local operations in multiple sub-spaces; results from these local operations are fused to come up with semantic relatedness estimates. Manifold DSM are constructed starting from a pairwise word-level semantic similarity matrix. The proposed model is evaluated on semantic similarity estimation task significantly improving on the state-of-the-art.